Overview of the INEX 2007 Entity Ranking Track

Many realistic user tasks involve the retrieval of specific entities instead of just any type of documents. Examples of information needs include ‘Countries where one can pay with the euro’ or ‘Impressionist art museums in The Netherlands’. The Initiative for Evaluation of XML Retrieval (INEX) started the XML Entity Ranking track (INEXXER) to create a test collection for entity retrieval in Wikipedia. Entities are assumed to correspond to Wikipedia entries. The goal of the track is to evaluate how well systems can rank entities in response to a query; the set of entities to be ranked is assumed to be loosely defined either by a generic category (entity ranking) or by some example entities (list completion). This track overview introduces the track setup, and discusses the implications of the new relevance notion for entity ranking in comparison to ad hoc retrieval.